Sparsity-Aware Learning and Compressed Sensing: An Overview

The notion of regularization has been widely used as a tool to address a number of problems that are usually encountered in Machine Learning. Improving the performance of an estimator by shrinking the norm of the MVU estimator, guarding against overfitting, coping with ill-conditioning, providing a solution to an underdetermined set of equations, are some notable examples where regularization has provided successful answers. A notable example is the ridge regression concept, where the LS loss function is combined, in a tradeoff rationale, with the Euclidean norm of the desired solution. In this paper, our interest will be on alternatives to the Euclidean norms and in particular the focus will revolve around the `1 norm; this is the sum of the absolute values of the components comprising a vector. Although seeking a solution to a problem via the `1 norm regularization of a loss function has been known and used since the 1970s, it is only recently that has become